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ABSTRACT 

Statistical invariance procedures provide a way of 
looking at the generalizability of research results from sample to 
sample when the research has not been validated by replication. This 
paper discusses the Procrustean Rotation invariance procedure 
following a canonical correlation analysis. The computer program 
RELATE is used to gauge the replicability and generalizability of the 
illustrative substantive research results. A Procrustean rotation 
forces orthogonal (uncorrelated) functions of factors to a best fit 
position after setting the factor vectors to unit length (l.o) in 
order to equalize the contribution of each factor vector to the 
determination of the amount of rotation necessary. This rotation 
technique can be used as a cross-validation procedure, splitting data 
from a single sample and comparing factor vectors for each half. The 
sample used to illustrate the procedure is from a study of leadership 
styles conducted by M. L. Tucker (1990) using data from 106 college 
faculty and administrators, with the university research sample split 
into two uneven data sets (n=48 and n=58) . Two tables of illustrative 
data and an ll-item list of references are included. (SLD) 
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ABSTRACT 



Thoughtful researchers place importance on replicability of research 
results. Statistical invariance procedures provide a way of looking at 
the general izability of research results from sample to sample when the 
research has not been validated by replication. The present paper pro- 
vides a discussion of the Procrustean Rotation invariaice procedure fol- 
lowing a canonical correlations analysis. The computer program RELATE is 
utilized to gauge the replicability and general izability of the illustra- 
tive substantive research results. 
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Researchers today still rely heavily on statistical significance 
testing to validate research findings. Yet, Tukey (1969) reminds re- 
searchers that it was never the intention of the statistical significance 
analysis to be substituted for replication of research. 

The modern test of significance, before which so many 
editors of psychological journals are reported to bow 
down, owes more to R. A. Fisher than to any other man. 
Yet Sir Ronald's standard of firm knowledge was not one 
very extremely significant result but rather the ability 
to repeatedly get results significant at 5%. Repetition 
is the basis for judging variability and significance and 
confidence. Repetition of results, each significant, is 
the basis, according to Fisher, of scientific truth, (p. 85) 
Yet all research cannot be replicative ones. There is a need, then, 
to address invariance of initial research findings. Kerlinger (1986) re- 
minds researchers that all research using sampling techniques is inher- 
ently subjected to sampling error. Even though survey information is gen- 
erally found to be relatively accurate, "there is always the one chance in 
twenty or a hundred that an error. . .caused by minor fluctuations of chance 
may occur (p. 387). It is pertinent, then, that sample invariance issues 
be addressed. Thompson describes invariance procedures as looking at how 
general izable results are from sample to sample, i.e., how much they do 
not vary from sample to sample. The logic of invariance analysis was sum- 
marized by Fish (1986), as an attempt to determine how stable the statis- 
tical results are likely to be across different samples. 

In the typical invariance procedure an analysis is performed 
separately on each of two subgroups into which the study 
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sample has been divided, and the results are compared. When 
the results of an analysis are not comparable--i .e. , not 
invariant—serious doubts about the general izability of the 
results are in order, (pp. 65-66) 
Invariance analysis provides more confidence that research results 
are stable and replicable across samples. The current study applied an 
invariance technique following a canonical correlation analysis. For 
readers unfamiliar with canonical correlation analysis, Thompson (1984) 
provides a very understandable explanation. 

Like all correlational methods, canonical correlations analysis tends 
to capitalize on chance; in fact, Thompson (1981) asserts that most ex- 
traction methods achieve their criterion by such capitalization. For a 
factor analysis, Gorsuch (1983) recommends that, by randomly splitting a 
study sample in half and comparing the factors extracted from both data 
sets, an estimate of the magnitude of this capitalization can be found. 

Procrustean Rotation 
A Procrustean rotation can be used with any multivariate technique. 
The name "Procrustean" originates in Greek mythology. Procrustes, a son 
of Poseidon, forced travelers spending the night at his home to fit his 
bed by either cutting off their legs or stretching their bodies. 
Similarly, a Procrustean rotation forces orthogonal (uncorrelated) func- 
tions of factors to a "best fit" position after setting the factor vectors 
to unit length (1.0) "in order to equalize the contribution of each 
[factor vector] to the determination of the amount of rotation necessary" 
(Veldman, 1967, p. 238). This rotation technique can be used as a 
cross-validation procedure, splitting the data from a single sample and 
comparing the factor vectors from each half. 
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The use of factor analysis as a means of validity evaluation is well 
known. Thompson and Pitts (1981/1982) describe this cosine application as 
a rotation of calculated factors to a position of "best fit" with a target 
matrix that has been theoretically derived. The target matrix determines 
the expected number of factors and the expected correlation between each 
item and each factor. Thus, the cosines of the angles between the actual 
and the hypothetical measures can be interpreted as validity coefficients. 

In this procedure, the original sample is split into two subsets, 
structure or function coefficients are generated, resulting in two sets of 
coefficients for comparison using the "best fit" rotation method. 
Thompson (1986) provides a detailed review of this empirical method devel- 
oped by Kaiser, Hunka, and Bianchini (1969) for "relating" factors derived 
from different samples of data. This method consists of projecting the 
two sets of factors into the same factor space and calculating the cosines 
of the angles among the factors across the two solutions. These cosines 
provide a measure of the relatedness of the two sets of factors, and are 
similar to correlation coefficients. 

Thompson (lQdl, 1986) affirms that these Procrustean rotation derived 
coefficients are analogous to test-retest coefficients and has called them 
invariance coefficients. They may also be utilized as adequacy coeffi- 
cients for substantive interpretations. 

Application of the Procrustean Rotation 

The sample used for illustration of the Procrustean rotation is taken 
from a substantive study by Tucker (1990) of transformational, transac- 
tional, and laissez-faire leadership styles as predictors of follower 
satisfaction with leader, of follower perceptions of leader effectiveness, 
and of extra effort expended by the follower for a leader. Data were 
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gathered from 106 faculty and administrative respondents who rated their 
individual leaders located at a Southern, urban university (XYZ). 

The Procrustean rotation of the standardized canonical function 
coefficients was utilized as an invariance procedure to further analyze 
the data at hand for general izabil ity. In particular, this research in- 
vestigated the similarity of function equations for the predictor variable 
set across subsamples of XYZ research data. Although either function co- 
efficients or structure coefficients may be submitted to a Procrustean 
rotation, for this particular application, function coefficients are used 
because the primary concern was to investigate the invariance of the pre- 
dictor variable set, focusing on the invariant predictability of function 
equations used to produce variate scores. 

This university research sample (N = 106) was split into two uneven 
data sets (n = 48, n = 58). Data from each subsample were entered into a 
separate canonical correlation analysis. The resulting predictor variable 
function coefficient matrices for Functions I, II, and III were submitted 
to a Procrustean rotation using RELATE {Veldman, 1967), as illustrated in 
Table 1. The standardized function coefficients for sample one (n = 48) 
were input as matrix A, and those from sample two as matrix B. The deci- 
sion regarding which matrix is to be designated the target for "best fit" 
rotation is arbitrary. The cosines, or correlation coefficients, that 
were derived when projecting the two sets of data into the same factor 
space, provide the relatedness of the two separate subsamples. Thompson 
(1981, 1984, 1986) affirms that these coefficients are analogous to 
invariance coefficients and states that they may also be used for substan- 
tive interpretations as adequacy coefficients. 



INSERT TABLE 1 ABOUT HERE 



Although the main focus of interest is the resulting matrix cosines, 
test r's should be first consulted. These evaluate the relation of the 
given variables from the two data sets within the factor space, and must 
be suitable for the functions being rotated to "best fit" to be suitable. 
Reliability of the correlation of each canonical function, based on the 
two matrices, was provided in test r's by RELATE. The test variable r's, 
as shown in Table 2 for the three rotated matrices, produced by RELATE 
were .997, .998, and .998, These results indicate (since they are large) 
that the three canonical functions can be rotated in this manner. 

With respect to the resulting cosines among the functions, generally, 
to be considered replicated, functions should have a cosine of roughly .8 
or higher (Thompson & Pitts, 1981/1982), Kaiser et al (1969) suggests .85 
is reasonable and Gorsuch (1983) recommends results greater than .93 as 
exceptional; however, Thompson (1986) presents empirically derived cutoffs 
as an alternative to the theoretically derived cutoffs formulated by oth- 
ers. 

The cosines among each of the three functions for this canonical cor- 
relation analysis data are presented in Table 2. The cosines resulting 
from the Procrustean-rotated canonical standardized functions were all 
close to one, above the .93 exceptional range set by Gorsuch, thus con- 
firming that the original results are sample "invariate." These 
Procrustean rotation invariance analysis results affirmed the invariance 
of this set of data for all three predictor variables on all three ca- 
nonical functions. 
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INSERT TABLE 2 ABOUT HERE 

This RELATE procedure produced an investigation of the similarity of 
function equations used to produce variate scores. The use of structure 
coefficients would have investigated "similarity of the distribution of 
variable variances across equations" (Thompson, 1934, p. 45). 

However, Thompson (1984) reminds us that there is much yet to le.rn 
about the characteristics of various invariance estimates and suggests 
that researchers employ several procedures in obtaining an "upper bound" 
and "lower bound" of the degree of capitalization on sampling specificity. 
Thompson (p. 46) cautions of the importance t recognize that all proce- 
dures are "liberal" estimates of invariance when one data set is split 
into subgroups, because the. . .subgroups came from the same sample and the 
subgroups and their parameter estimates are therefore interdependent. A 
more conservative approach to the estimation of result invariance is pro- 
vided when the canonical analysis involves two independent samples of sub- 
jects. 

Nevertheless, it is imperative to derive some empirical evaluation of 
the general izability of results, since the cumulation of findings across 
studies is the business of science. 
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Table 1 

Matrices Entered Into Procrustean Rotation: 

Standardised Func tion Coefficients of Each Split-Sample Canonical R.m 

MATRIX A (TARGET MATRIX, n = 48): 

Function I Function II Function III 

0.98000 1.04100 0.64000 

0.00700 1.34600 0.01500 

0.02900 0.40000 1.17300 

MATRIX B (n = 58): 

Function I Function II Function III 

0.99200 0.80700 0.61100 

0.00100 1.29900 0.38200 

0.02100 0.16200 1.14800 



Table 2 

Cosines Among Factor Ay es Resulting From Procrustean 
Rotation Inva riance Proc edure, Wi th Test r' s 

Function I Function II 

Function I 0.9978 0T0006~ 

Function II 0.0145 0.9776 

Function III 0.0645 0.2103 

Note. TEST Ps were 0.9971, 0.9979, and 0.9979. 

1 0 
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Function III 
0.0661 
0.2098 
0.9755 



